Record wrapper command context and errors without input snapshots - #5621

The wrapper now records selected errors, command context, and child process outcomes in BCS spans. Expanded end-to-end tests check these records and confirm that full inputs and returned-value snapshots are excluded. The README and TELEMETRY.md describe capture boundaries and limitations.

Changes: Wrapper telemetry capture

Layer / File(s)	Summary
Record errors and toolchain outcomes baml_language/wrapper/baml_src/entry.baml, baml_language/wrapper/baml_src/invocation.baml, baml_language/wrapper/baml_src/selection.baml, baml_language/wrapper/baml_src/selectors.baml	Selected spans now record errors. Dispatch spans include bounded command metadata. Toolchain execution returns the child exit code and signal, and dispatch exits with the child exit code.
Verify and document capture behavior baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs, baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs, baml_language/wrapper/README.md, TELEMETRY.md	End-to-end tests check errors, statuses, command context, telemetry settings, and omitted input and output values. Documentation describes span boundaries, process outcomes, sensitive error payloads, and capture limitations.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Dispatch
  participant run_toolchain
  participant ChildBinary
  Dispatch->>run_toolchain: Run toolchain command
  run_toolchain->>ChildBinary: Launch with BAML_CLI_ALLOW_DIRECT=1
  ChildBinary-->>run_toolchain: Return exit code and signal
  run_toolchain-->>Dispatch: Return process outcome
  Dispatch->>Dispatch: Refresh channel and exit with child exit code

Loading

Possibly related issues

• A telemetry level that records timing and token usage but never prompt or response bodies #5249 — The PR narrows wrapper capture boundaries but does not add a metadata-only telemetry level.
• Feature request: trace to Boundary without the prompt (request bodies / rendered prompt), keeping metadata and outputs #5309 — The PR excludes wrapper input snapshots but does not remove HTTP request bodies from network spans.

Possibly related PRs

• BoundaryML/baml#5624 — This PR implements the same wrapper command metadata behavior, documentation updates, and command-context test.
• BoundaryML/baml#5491 — This PR changes the embedded telemetry recording setup that the packed-wrapper tests exercise.
• BoundaryML/baml#5422 — This PR changes wrapper functions and files introduced by that PR.

Suggested reviewers: hellovai, aaronvg, 2kai2kai2

Merge Risk: Minimal · up to 049b4

The wrapper now records errors and delegated process outcomes without capturing input snapshots. The captured-argument policy is documented and has an opt-out. No concrete merge-blocking risk was identified.

Security architecture risk: Moderate · up to 049b4

Complete errors can contain sensitive command arguments, selectors or paths, and release recordings can send those values to the publisher’s cloud project. Disabling input snapshots does not remove that exposure. Command labels and child results remain narrowly bounded, and recording can be disabled.

Retained concerns

• Medium · security · inferred: New full-error capture creates a path for sensitive input-derived values into default cloud recordings. Parser errors interpolate project paths, and management errors retain raw arguments. Release recording uses the existing publisher-owned destination unless overridden or disabled, so suppressing input snapshots does not prevent secrets embedded in failing inputs from leaving the invoking environment. This is a newly expanded payload boundary, not a newly introduced destination or a demonstrated public read vulnerability.

Security review details

Security Blast Radius

• inferred — The affected population is enabled BAML-native wrapper recordings, including release builds using embedded ingestion. Exposure is limited to values present in captured errors, not wholesale argv, environment or child output. Multiple users’ errors can reach the selected recording project; caller-owned ingestion overrides can change that destination. Reader privileges and retention at the cloud destination remain unverified.

Security Findings and Attack Paths

• inferred — A sensitive value supplied as an invalid management argument or project path can enter a wrapper error and then a recording. The test explicitly expects an unrecognized argument to survive in error_value despite absent input_args. In release cloud-recording configurations this supplies a disclosure path to the recording destination; neither an actual secret nor unauthorized downstream access was demonstrated.

Trust Boundaries and Controls

• observed — The decisive boundary is local diagnostic content entering persisted runtime recordings, distinct from metadata-only PostHog events. Input suppression, narrow process results and HTTP-body exclusion constrain collection but do not redact error payloads. BAML_TELEMETRY=off is the documented recording control; tests demonstrate local recording with DO_NOT_TRACK=1, so that PostHog control must not be treated as disabling runtime recordings.

Resilience and Maintainability Implications

• observed — The new result boundary retains child status before dispatch exits and excludes the full subprocess result. Existing recovery handlers remain in place, including successful wrapper-only version reporting after recoverable preparation or launch errors. These controls preserve failure containment while making recovered errors queryable.

Hardening Proposals

• proposed — For publisher-owned recordings, prefer an allowlisted error projection containing classification and bounded context. Make unrestricted error values a separate explicit diagnostic opt-in rather than relying on input-snapshot suppression.
• proposed — Before broad release of full-error capture, establish the destination’s reader scope, retention, deletion ownership and response procedure for accidentally recorded credentials. Disabling future recording should not substitute for handling values already delivered.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title names the main change: wrapper command context and error recording, with full input snapshots removed. The changeset adds bounded canonical command context to foreground dispatch spans, reco…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.

Full details: Title check

Explanation

The title names the main change: wrapper command context and error recording, with full input snapshots removed. The changeset adds bounded canonical command context to foreground dispatch spans, records error payloads at wrapper operational spans, and disables input and output capture. The title matches these points and stays concise.

Generate unit tests (beta)

• Commit to this branch
• Create a new PR

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Script executed:

#!/bin/bash
set -u
revision=cb461efc996ebdd5b2e88814f17985a78d11c098
printf '%s\n' '=== Local revision ==='
git rev-parse HEAD
git show -s --format='%H %s' "$revision" 2>&1 || true
printf '%s\n' '=== Current PR head ==='
gh pr view 5621 --repo BoundaryML/baml --json headRefOid --jq .headRefOid 2>&1 || true
printf '%s\n' '=== Wrapper context references at requested revision ==='
git grep -n -C 3 -E 'command|posthog|PostHog|trace|run_toolchain' "$revision" -- baml_language/wrapper/baml_src/entry.baml baml_language/wrapper/baml_src/invocation.baml | head -n 300 || true
printf '%s\n' '=== Additional command-context definitions ==='
git grep -n -C 3 -E 'foreground|canonical|set.*command|clear.*command' "$revision" -- baml_language/wrapper baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs | head -n 220 || true

Length of output: 39334

---

Script executed:

#!/bin/bash
set -u
revision=cb461efc996ebdd5b2e88814f17985a78d11c098
head=a76b8349d33e7b17b914aaa2003d97ec53b36e00
printf '%s\n' '=== Ancestry ==='
if git merge-base --is-ancestor "$revision" "$head"; then
  echo 'Requested revision is an ancestor of the current PR head.'
else
  echo 'Requested revision is not an ancestor of the current PR head.'
fi
printf '%s\n' '=== Later commits ==='
git log --oneline "$revision..$head"
printf '%s\n' '=== Changes since requested revision ==='
git diff --stat "$revision" "$head"
git diff --name-status "$revision" "$head"

Length of output: 1536

---

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-12T23:37:21.402Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/entry.baml"]: In the BAML wrapper's baml_language/wrapper/baml_src/entry.baml, foreground dispatch uses bounded canonical command/subcommand context independently of PostHog, and internal helpers clear inherited foreground command labels. When reviewing dispatch tracing, check both invariants.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Script executed:

#!/bin/bash
set -u
requested=35372e935092f95ecb1ab7bb34a2c4856e5b2995
echo '=== PR head and checks ==='
gh pr view 5621 --repo BoundaryML/baml --json headRefOid,baseRefOid,mergeStateStatus,statusCheckRollup --jq '{headRefOid,baseRefOid,mergeStateStatus,checks: [.statusCheckRollup[]? | {name,context,status,conclusion,state}]}' 2>&1
echo '=== Local revision ==='
git rev-parse HEAD
git show -s --format='%H %s' "$requested" 2>&1
echo '=== Changed files and summary ==='
base=$(gh pr view 5621 --repo BoundaryML/baml --json baseRefOid --jq .baseRefOid 2>/dev/null)
if [ -n "$base" ] && git cat-file -e "$base^{commit}" 2>/dev/null && git cat-file -e "$requested^{commit}" 2>/dev/null; then
  git diff --stat "$base...$requested"
  echo '=== Wrapper implementation diff ==='
  git diff --no-ext-diff --unified=5 "$base...$requested" -- baml_language/wrapper/baml_src/entry.baml baml_language/wrapper/baml_src/invocation.baml
  echo '=== Documentation diff ==='
  git diff --no-ext-diff --unified=3 "$base...$requested" -- TELEMETRY.md baml_language/wrapper/README.md
else
  echo 'Unable to inspect the requested revision against the PR base.'
fi
echo '=== Test structure ==='
ast-grep outline baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs --items all --type function 2>&1
ast-grep outline baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs --items all --type function 2>&1
echo '=== Signal regression references ==='
rg -n -C 5 'SIGTERM|signal.?15|exit_code.?143|cancelled|canceled' baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs || true

Length of output: 28094

---

Script executed:

#!/bin/bash
set -u
base=50329b7dd1f60b096b3be5586945a18eac595bd6
head=35372e935092f95ecb1ab7bb34a2c4856e5b2995
echo '=== Acquisition and passive boundary changes ==='
git diff --no-ext-diff --unified=10 "$base...$head" -- baml_language/wrapper/baml_src/selection.baml baml_language/wrapper/baml_src/selectors.baml
echo '=== Dispatch and helper logic ==='
cat -n baml_language/wrapper/baml_src/entry.baml | sed -n '1,205p'
echo '=== Command normalization references ==='
rg -n -C 5 'function command\(|function is_internal|__.*telemetry|__.*replace' baml_language/wrapper/baml_src
echo '=== Signal regression ==='
rg -n -i -C 8 'signal|kill|143|termination|terminated' baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs || true
echo '=== Tests: fixture and assertions ==='
cat -n baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs | sed -n '792,1305p'
echo '=== Context regression assertions ==='
cat -n baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs | sed -n '1,390p'

Length of output: 71212

---

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-12T23:43:12.611Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/entry.baml","baml_language/wrapper/baml_src/invocation.baml"]: When reviewing explicit wrapper operational spans in baml_language/wrapper/baml_src/entry.baml or baml_language/wrapper/baml_src/invocation.baml, full typed error payloads are intentional and may include offending arguments, selectors, paths, or secrets. Assess sensitive-data exposure and verify that user-facing documentation warns about secrets in errors. Do not require redaction solely because an error quotes an offending input.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-12T23:37:23.520Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/entry.baml"]: In the BAML wrapper's baml_language/wrapper/baml_src/entry.baml, foreground dispatch uses bounded canonical command/subcommand context independently of PostHog, and internal helpers clear inherited foreground command labels. When reviewing dispatch tracing, check both invariants.

Changes approved. Review threads were not changed.

---

Full review finished.

Script executed:

#!/bin/bash
set -u
requested=049b4f221548a66b1d09557a993051c5edcaa757
previous=35372e935092f95ecb1ab7bb34a2c4856e5b2995
echo '=== PR head, base, merge state, checks ==='
gh pr view 5621 --repo BoundaryML/baml --json headRefOid,baseRefOid,mergeStateStatus,statusCheckRollup --jq '{headRefOid,baseRefOid,mergeStateStatus,checks:[.statusCheckRollup[]? | {name,context,status,conclusion,state}]}' 2>&1 || true
echo '=== Local and requested revisions ==='
git rev-parse HEAD
git show -s --format='%H %s' "$requested" 2>&1 || true
echo '=== Diff from previously reviewed head ==='
if git cat-file -e "$requested^{commit}" 2>/dev/null && git cat-file -e "$previous^{commit}" 2>/dev/null; then
 git diff --stat "$previous" "$requested"
 git diff --name-status "$previous" "$requested"
 echo '=== Version behavior and telemetry diff ==='
 git diff --no-ext-diff --unified=8 "$previous" "$requested" -- baml_language/wrapper/baml_src/entry.baml baml_language/wrapper/baml_src/invocation.baml baml_language/wrapper/baml_src/selection.baml baml_language/wrapper/baml_src/selectors.baml
 echo '=== Test changes ==='
 git diff --no-ext-diff --unified=5 "$previous" "$requested" -- baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs baml_language/crates/baml_cli/tests/wrapper_command_context_e2e.rs
fi
echo '=== Outlines ==='
ast-grep outline baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs --items all --type function --match 'wrapper|telemetry|record' 2>&1 || true
echo '=== Version call chain and recovery ==='
rg -n -C 5 'run_cli_version|print_version|run_toolchain|ensure_installed|warn_if_outdated|refresh_channel' baml_language/wrapper/baml_src/entry.baml baml_language/wrapper/baml_src/invocation.baml baml_language/wrapper/baml_src/selection.baml baml_language/wrapper/baml_src/selectors.baml || true

Length of output: 46995

---

Script executed:

#!/bin/bash
set -u
base=8bae9696a8e21da3417d0b37aea3a0a63db4bb55
head=049b4f221548a66b1d09557a993051c5edcaa757
echo '=== Actual PR changed files ==='
git diff --stat "$base...$head"
git diff --name-status "$base...$head"
echo '=== Wrapper version upstream vs PR ==='
git show "$base":baml_language/wrapper/baml_src/invocation.baml | nl -ba | sed -n '290,350p'
cat -n baml_language/wrapper/baml_src/invocation.baml | sed -n '300,340p'
echo '=== Current PR wrapper diff ==='
git diff --no-ext-diff --unified=5 "$base...$head" -- baml_language/wrapper/baml_src/entry.baml baml_language/wrapper/baml_src/invocation.baml baml_language/wrapper/baml_src/selection.baml baml_language/wrapper/baml_src/selectors.baml
echo '=== Version regression fixtures and checks ==='
cat -n baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs | sed -n '792,1005p'
cat -n baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs | sed -n '1080,1205p'
cat -n baml_language/crates/baml_cli/tests/embedded_telemetry_e2e.rs | sed -n '1250,1355p'
echo '=== Security and version documentation ==='
rg -n -i -C 3 'secret|record|snapshot|error|version|process exit|telemetry=off' TELEMETRY.md baml_language/wrapper/README.md | head -n 260

Length of output: 72971

---

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-13T00:17:00.122Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/invocation.baml"]: In baml_language/wrapper/baml_src/invocation.baml, selected-CLI version reporting is fail-open for recovered selection, acquisition, and launch failures: the wrapper reports a successful process outcome. When the selected CLI runs, run_cli_version returns its exit_code from run_toolchain and print_version passes that code to sys.exit. Preserve both behaviors when reviewing version-reporting changes.

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-13T00:17:00.122Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/invocation.baml"]: In baml_language/wrapper/baml_src/invocation.baml, toolchain list must not acquire a toolchain. When reviewing list behavior, check that it remains a local inventory operation without acquisition.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-12T23:43:13.694Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/entry.baml","baml_language/wrapper/baml_src/invocation.baml"]: When reviewing explicit wrapper operational spans in baml_language/wrapper/baml_src/entry.baml or baml_language/wrapper/baml_src/invocation.baml, full typed error payloads are intentional and may include offending arguments, selectors, paths, or secrets. Assess sensitive-data exposure and verify that user-facing documentation warns about secrets in errors. Do not require redaction solely because an error quotes an offending input.

Learnt from: sxlijin
URL: https://github.com/BoundaryML/baml/pull/5621

Timestamp: 2026-11-12T23:37:23.520Z
Learning: Within repository "BoundaryML/baml", only for files matching one of ["baml_language/wrapper/baml_src/entry.baml"]: In the BAML wrapper's baml_language/wrapper/baml_src/entry.baml, foreground dispatch uses bounded canonical command/subcommand context independently of PostHog, and internal helpers clear inherited foreground command labels. When reviewing dispatch tracing, check both invariants.

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.